U.S. Pat. No. 6,173,259 describes a system wherein speech recognition tasks are produced from a plurality of terminals and processed by a plurality of processors. A server operates as a task distribution device that receives the tasks from the terminals, queues the tasks and distributes the tasks from the queues to the processors. Thus, the processors can be efficiently used because any processor may receive tasks generated by any terminal when it is free. The tasks can be evenly distributed over the processors, to minimize the delay before any task can start . . . .
The task distribution device may form a bottleneck in the operation of the system. When the task distribution device receives too many tasks simultaneously, the task distribution device could create a delay for accepting new tasks. For relatively big tasks, which take much more time to execute than to transmit to a processor this is hardly a problem, because the processors will block processing of new tasks before the server when there are many tasks. However in a highly distributed environment wherein small tasks are processed such a bottleneck can be significant. For example, in an integrated circuit wherein many small tasks are handled by small sub-circuits such a bottleneck could arise.
Various approaches may be used to solve this problem. In one approach each task source may be coupled to its own set of one or more processors. However, this results in inefficient use of processors for example when one task source produces many tasks and other task sources produce few tasks. In another approach a plurality of task distribution devices may be used in parallel. In a simple implementation each task distribution device is coupled to its own set of task sources and task processors. In this way the bottlenecks formed by each task distribution device can be limited, but the processors may still not be used optimally, for example when many tasks are handled by one task distribution device and few by other task distribution devices. To improve on this multiplexing circuits may be used that allow multiple task distribution devices to receive tasks from the same sources or to send tasks to the same processors. However, this reintroduces the risk of bottlenecks.
From the art of telecommunication it is known to route message that are addressed to specific destinations via alternative routes. The network selects between the alternative routes dependent on the availability of routers in the alternative routes. Eventually the messages are routed to the intended destination. Such systems do not work with messages that have no specific destination.